{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f802e64d-4eaa-445d-a48a-1042a91bc394",
   "metadata": {
    "tags": []
   },
   "source": [
    "# 基于AgentRuntime服务化组件\n",
    "\n",
    "## 目标\n",
    "使用 AgentRuntime 对组件进行服务化。\n",
    "\n",
    "AgentRuntime 是对组件（Component）的服务化封装，具体有如下几个功能：\n",
    "- 一键服务化组件: 使得组件能够以服务的形式运行，支持 API 调用和对话框交互。\n",
    "- Session 数据管理: 提供 Session 数据的管理功能，允许跟踪和存储用户会话数据。\n",
    "- 请求时鉴权: 支持在请求时进行认证，确保安全性。\n",
    "\n",
    "\n",
    "## 准备工作\n",
    "### 安装Python SDK\n",
    "\n",
    "appbuilder 支持使用 pip 安装（要求Python >= 3.8），并且 AgentRuntime 服务化组件依赖 `appbuilder-sdk[serve]`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2939356f-61c2-42e9-9e0c-fc6729c193f6",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "pip install appbuilder-sdk 'appbuilder-sdk[serve]'"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "aeb2fa55-075f-48df-a9fb-8b40d9900684",
   "metadata": {},
   "source": [
    "## 基本用法\n",
    "\n",
    "### 快速开始\n",
    "\n",
    "下面的示例会基于 Playground 组件，在 8091 端口部署 Web 服务: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41559341-fd7a-478c-a08b-1477d79e9d41",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-12-18T06:24:26.982459Z",
     "start_time": "2023-12-18T06:23:53.771345Z"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import appbuilder\n",
    "\n",
    "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n",
    "os.environ[\"APPBUILDER_TOKEN\"] = '...'\n",
    "\n",
    "component = appbuilder.Playground(\n",
    "    prompt_template=\"{query}\",\n",
    "    model=\"ERNIE-Bot\"\n",
    ")\n",
    "\n",
    "agent = appbuilder.AgentRuntime(component=component)\n",
    "agent.serve(port=8091)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b71e24eb",
   "metadata": {},
   "source": [
    "通过 Shell 命令测试启动的服务, 请求 Body 为组件 run 方法的入参：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12c71fa5",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "curl --location 'http://0.0.0.0:8091/chat' \\\n",
    "--header 'Content-Type: application/json' \\\n",
    "--data '{\n",
    "    \"message\": \"海淀区的面积是多少\",\n",
    "    \"stream\": false\n",
    "}'"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5fc5bc38-6bc5-4187-a8fd-f802d77d89fa",
   "metadata": {},
   "source": [
    "## AgentRuntime 参数说明"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2364c35",
   "metadata": {},
   "source": [
    "### 1. 类初始化参数说明\n",
    "\n",
    "AgentRuntime 初始化接受两个参数。\n",
    "\n",
    "| 参数名称 | 参数类型 | 是否必须 | 描述 | 示例值 |\n",
    "|--|--|--|--|--|\n",
    "| component | Component | 是 | 可运行的 Component, 该 Component 需要实现 run(message, stream, **args) 方法。 | Playground(prompt_template=\"{query}\", model=\"ERNIE-Bot\") |\n",
    "| user_session_config | sqlalchemy.engine.URL\\|Str\\|None | 否 | 会话 Session 数据存储的数据库配置，遵循 sqlalchemy 后端定义，可参考[文档](https://docs.sqlalchemy.org/en/20/core/engines.html#backend-specific-urls)。默认使用 sqlite:///user_session.db，即本地的 SQLite 存储 | \"sqlite:///user_session.db\" |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62e1af06",
   "metadata": {},
   "source": [
    "### 2. API 服务参数\n",
    "\n",
    "#### 2.1 请求参数\n",
    "\n",
    "**接口定义**\n",
    "\n",
    "| URL | Method |\n",
    "|--|--|\n",
    "| /chat | POST |\n",
    "\n",
    "**Header 参数**\n",
    "\n",
    "| 参数名称 | 是否必须 | 描述 | 示例值 |\n",
    "|--|--|--|--|\n",
    "| Content-Type | 是 | 必须设置为\"application/json\" | \"application/json\" |\n",
    "| X-Appbuilder-Token | 否 | 开启请求时认证能力时需要带入 APPBUILDER_TOKEN 进行鉴权 | 前往千帆AppBuilder官网创建密钥，流程详见[文档](https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥) |\n",
    "\n",
    "**Body 参数**\n",
    "\n",
    "| 参数名称 | 参数类型 | 是否必须 | 描述 | 示例值 |\n",
    "|--|--|--|--|--|\n",
    "| message | Any | 是 | 透传到 component 的 run 方法的 message 参数 | \"海淀区的面积是多少\" |\n",
    "| stream | Bool | 否 | 是否流式调用。透传到 component 的 run 方法的 stream 参数。默认为 false | false |\n",
    "| session_id | Str | 否 | 用于标示同一个会话（Session）。如果不传该值，后端会自动生成 session_id，在响应参数中返回 | \"99680089-5acb-4298-9ade-a1a3f6c28102\" |\n",
    "| 其他参数 | Any | 否 | 透传到 component 的 run 方法 | - |\n",
    "\n",
    "\n",
    "#### 2.2 响应参数\n",
    "分为非流式响应和流式响应。\n",
    "\n",
    "**非流式响应**\n",
    "\n",
    "| 参数名称 | 参数类型 | 描述 | 示例值 |\n",
    "|--|--|--|--|\n",
    "| code | Int | 错误码。值为0表示成功，否则为失败。非0错误详见错误码部分描述 | 0 |\n",
    "| message | Str | 错误信息描述。 | \"Missing input variable query in message ['海淀区的面积是多少']\" |\n",
    "| result | Object | 请求结果 | - |\n",
    "| + answer_message | Object | 组件返回值，由返回的 Message 序列化得到 | {\"content\":\"海淀区是北京市的一个区，位于北京市主城区西部和西北部，东与西城区、朝阳区相邻，南与丰台区毗连，西与石景山区、门头沟区交界，北与昌平区接壤。海淀区的面积为**431平方千米**，约占北京市总面积的2.6%。\",\"extra\":{},\"id\":\"6b4e5019-a708-4bc5-a6ec-595fb4285677\",\"mtype\":\"dict\",\"name\":\"msg\"} |\n",
    "| + session_id | Str | 用于标示同一个会话（Session） | \"99680089-5acb-4298-9ade-a1a3f6c28102\" |\n",
    "\n",
    "**流式响应**\n",
    "\n",
    "流式数据以追加的形式返回。流式和非流式的数据结构一致，不再描述。\n",
    "\n",
    "#### 2.3 响应示例\n",
    "\n",
    "分为非流式响应和流式响应。\n",
    "\n",
    "**非流式响应**\n",
    "\n",
    "```shell\n",
    "{\n",
    "  \"code\": 0,\n",
    "  \"message\": \"\",\n",
    "  \"result\": {\n",
    "    \"answer_message\": {\n",
    "      \"content\": \"海淀区是北京市的一个区，位于北京市主城区西部和西北部，东与西城区、朝阳区相邻，南与丰台区毗连，西与石景山区、门头沟区交界，北与昌平区接壤。海淀区的面积为**431平方千米**，约占北京市总面积的2.6%。\",\n",
    "      \"extra\": {},\n",
    "      \"id\": \"6b4e5019-a708-4bc5-a6ec-595fb4285677\",\n",
    "      \"mtype\": \"dict\",\n",
    "      \"name\": \"msg\"\n",
    "    },\n",
    "    \"session_id\": \"99680089-5acb-4298-9ade-a1a3f6c28102\"\n",
    "  }\n",
    "}\n",
    "```\n",
    "\n",
    "**流式响应**\n",
    "\n",
    "```shell\n",
    "data: {\"code\": 0, \"message\": \"\", \"result\": {\"session_id\": \"663303a9-d83d-481f-a084-872ece87989c\", \"answer_message\": {\"content\": \"海淀区\", \"extra\": {}}}}\n",
    "\n",
    "data: {\"code\": 0, \"message\": \"\", \"result\": {\"session_id\": \"663303a9-d83d-481f-a084-872ece87989c\", \"answer_message\": {\"content\": \"，隶属于北京市，位于北京市主城区西部和西北部，东与西城区、朝阳区相邻，南与丰台区毗连，\", \"extra\": {}}}}\n",
    "\n",
    "data: {\"code\": 0, \"message\": \"\", \"result\": {\"session_id\": \"663303a9-d83d-481f-a084-872ece87989c\", \"answer_message\": {\"content\": \"西与石景山区、门头沟区交界，北与昌平区接壤，总面积**431平方千米**。\", \"extra\": {}}}}\n",
    "\n",
    "data: {\"code\": 0, \"message\": \"\", \"result\": {\"session_id\": \"663303a9-d83d-481f-a084-872ece87989c\", \"answer_message\": {\"content\": \"\", \"extra\": {}}}}\n",
    "```\n",
    "\n",
    "#### 2.4 错误码\n",
    "| 错误码 | 描述 |\n",
    "|--|--|\n",
    "| 400 | 客户端请求参数错误 |\n",
    "| 1000 | 服务端执行错误 |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61923b00",
   "metadata": {},
   "source": [
    "## 高级用法"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2303c76",
   "metadata": {},
   "source": [
    "### 1. 一键服务化组件 \n",
    "AgentRuntime 可以快速组件以服务的形式运行，支持 API 调用和对话框交互。\n",
    "\n",
    "**1.1 API调用**\n",
    "\n",
    "API 调用的基础用法在快速开始小结已经给出，这里不再赘述。\n",
    "\n",
    "下面介绍使用 `gunicorn` 启动生产级 Web 服务的方法，`gunicorn` 是一个适用于 UNIX 的 Python WSGI HTTP 服务器，详见[项目链接](https://github.com/benoitc/gunicorn)。\n",
    "\n",
    "首先创建 `app.py` 文件，暴露 Flask App："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fde5cc94",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import appbuilder\n",
    "\n",
    "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n",
    "os.environ[\"APPBUILDER_TOKEN\"] = '...'\n",
    "\n",
    "def get_flask_app():\n",
    "    component = appbuilder.Playground(\n",
    "        prompt_template=\"{query}\",\n",
    "        model=\"ERNIE-Bot\"\n",
    "    )\n",
    "    agent = AgentRuntime(component=component)\n",
    "    return agent.create_flask_app()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a41ef57b",
   "metadata": {},
   "source": [
    "基于 `gunicorn` 启动生产级服务："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2077833f",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "# 服务工作进程数\n",
    "SERVER_WORKER_AMOUNT=8\n",
    "# 服务工作进程启动方式\n",
    "SERVER_WORKER_CLASS=gevent\n",
    "# 服务超时时间\n",
    "GUNICORN_TIMEOUT=60\n",
    "\n",
    "gunicorn \\\n",
    "  --bind \"0.0.0.0:8091\" \\\n",
    "  --workers ${SERVER_WORKER_AMOUNT} \\\n",
    "  --worker-class ${SERVER_WORKER_CLASS} \\\n",
    "  --timeout ${GUNICORN_TIMEOUT} \\\n",
    "  \"app:get_flask_app()\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "673c7565",
   "metadata": {},
   "source": [
    "**1.2 对话框交互**\n",
    "\n",
    "基于 chainlit 的对话框交互对被服务化的组件的 message 参数更加严格，要求能够接受 Str 的基础类型。\n",
    "\n",
    "执行下面的代码，会启动一个 chainlit 页面，页面地址：0.0.0.0:8091"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e485544f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import appbuilder\n",
    "\n",
    "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n",
    "os.environ[\"APPBUILDER_TOKEN\"] = '...'\n",
    "\n",
    "component = appbuilder.Playground(\n",
    "    prompt_template=\"{query}\",\n",
    "    model=\"ERNIE-Bot\"\n",
    ")\n",
    "\n",
    "agent = appbuilder.AgentRuntime(component=component)\n",
    "agent.chainlit_demo(port=8091)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d63bece",
   "metadata": {},
   "source": [
    "Chainlit Demo页面示意图如下所示，\n",
    "\n",
    "![chainlit demo](image/agent_runtime_with_chainlit_demo.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a074f367",
   "metadata": {},
   "source": [
    "**1.3 对话框交互AppBuilderClient**\n",
    "\n",
    "基于 chainlit 的对话框交互AppBuilderClient，可实现对话交互。\n",
    "\n",
    "执行下面的代码，会启动一个 chainlit 页面，页面地址：0.0.0.0:8091。可在页面上上传文档（可选）、执行对话。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a75e035",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import appbuilder\n",
    "\n",
    "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n",
    "os.environ[\"APPBUILDER_TOKEN\"] = \"...\"\n",
    "# 使用之前，在官网个人空间获取应用ID，如下图\n",
    "app_id= \"...\"\n",
    "agent_builder = appbuilder.AppBuilderClient(app_id)\n",
    "agent = appbuilder.AgentRuntime(component=agent_builder)\n",
    "agent.chainlit_agent(port=8091)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5162e29",
   "metadata": {},
   "source": [
    "在官网个人空间获取应用ID\n",
    "![get app_id](../app_builder_resources/app_id.png)\n",
    "\n",
    "使用服务上传文件并对话示意图如下所示\n",
    "![chainlit demo](./image/agent_runtime_with_chainlit_agent.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2392f46d",
   "metadata": {},
   "source": [
    "### 2. Session 数据管理\n",
    "AgentRuntime 提供 Session 数据的管理功能，允许跟踪和存储用户会话数据。一般只有在二次开发的组件需要使用该能力。\n",
    "\n",
    "**2.1 二次开发组件**\n",
    "\n",
    "二次开发的组件需要重写组件的 run(message, stream, **args)方法，并且至少需要有 message 和 stream 两个参数。\n",
    "\n",
    "下面基于 QueryRewrite 和 Playground 两个组件，开发 PlaygroundWithHistory 组件，该组件需要对会话数据进行操作。\n",
    "\n",
    "当使用 Component 独立运行时，会话数据会被存储于内存。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "079048e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import logging\n",
    "from appbuilder.core.component import Component\n",
    "from appbuilder import (\n",
    "    AgentRuntime, UserSession, Message, QueryRewrite, Playground,\n",
    ")\n",
    "\n",
    "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n",
    "os.environ[\"APPBUILDER_TOKEN\"] = '...'\n",
    "\n",
    "class PlaygroundWithHistory(Component):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.query_rewrite = QueryRewrite(model=\"ERNIE Speed-AppBuilder\")\n",
    "        self.playground = Playground(\n",
    "            prompt_template=\"{query}\",\n",
    "            model=\"ERNIE-Bot\"\n",
    "        )\n",
    "\n",
    "    def run(self, message: Message, stream: bool=False):\n",
    "        user_session = UserSession()\n",
    "        # 获取 Session 历史数据\n",
    "        history_queries = user_session.get_history(\"query\", limit=1)\n",
    "        history_answers = user_session.get_history(\"answer\", limit=1)\n",
    "\n",
    "        # query 改写\n",
    "        if history_queries and history_answers:\n",
    "            history = []\n",
    "            for query, answer in zip(history_queries, history_answers):\n",
    "                history.extend([query.content, answer.content])\n",
    "            logging.info(f\"history: {history}\")\n",
    "            message = self.query_rewrite(\n",
    "                Message(history + [message.content]), rewrite_type=\"带机器人回复\")\n",
    "        logging.info(f\"message: {message}\") \n",
    "\n",
    "        # 执行 playground\n",
    "        answer = self.playground.run(message, stream)\n",
    "\n",
    "        # 保存本轮数据\n",
    "        user_session.append({\n",
    "            \"query\": message,\n",
    "            \"answer\": answer,\n",
    "        }) \n",
    "        return answer\n",
    "\n",
    "# component 可以独立运行，session数据会被保存于内存\n",
    "playground_with_history_component = PlaygroundWithHistory()\n",
    "print(playground_with_history_component.run(Message(\"海淀区的面积是多少\"), stream=False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "480fd56d",
   "metadata": {},
   "source": [
    "**2.2 会话数据存储数据库**\n",
    "\n",
    "使用 AgentRuntime 对 Component 服务化，会话数据会被存储于数据库。\n",
    "下面的代码以 SQLite 为例展示该能力，更多数据库配置详见[文档](https://docs.sqlalchemy.org/en/20/core/engines.html#backend-specific-urls)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "111b11de",
   "metadata": {},
   "outputs": [],
   "source": [
    "user_session_config = \"sqlite:///foo.db\"\n",
    "agent = appbuilder.AgentRuntime(\n",
    "    component=playground_with_history_component, \n",
    "    user_session_config=user_session_config)\n",
    "agent.serve(port=8091)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7dc38ec",
   "metadata": {},
   "source": [
    "### 3. 请求时鉴权\n",
    "AgentRuntime 支持在请求时进行认证，确保安全性。\n",
    "\n",
    "使用该能力，在初始化组件时需要设置 lazy 鉴权："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2125bf4f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import appbuilder\n",
    "\n",
    "# 无需配置 APPBUILDER_TOKEN 环境变量\n",
    "\n",
    "component = appbuilder.Playground(\n",
    "    prompt_template=\"{query}\",\n",
    "    model=\"ERNIE-Bot\",\n",
    "    lazy_certification=True, # 设置 lazy 鉴权，在创建时不进行认证\n",
    ")\n",
    "\n",
    "agent = appbuilder.AgentRuntime(component=component)\n",
    "agent.serve(port=8091)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbf35a0a",
   "metadata": {},
   "source": [
    "当初始化组件时进行了 lazy 鉴权，请求时请求头必须带上 `X-Appbuilder-Token` （即Appbuilder密钥，获取流程详见[千帆AppBuilder官网创建密钥](https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥)）进行鉴权："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60dcfa37",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "curl --location 'http://0.0.0.0:8091/chat' \\\n",
    "    --header 'Content-Type: application/json' \\\n",
    "    --header 'X-Appbuilder-Token: ...' \\\n",
    "    --data '{\n",
    "        \"message\": \"海淀区的面积是多少\",\n",
    "        \"stream\": false\n",
    "    }'"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
